-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add options for zarr array definition #48
Conversation
Codecov Report
@@ Coverage Diff @@
## master #48 +/- ##
==========================================
+ Coverage 94.89% 95.00% +0.10%
==========================================
Files 10 10
Lines 392 400 +8
Branches 75 78 +3
==========================================
+ Hits 372 380 +8
Misses 10 10
Partials 10 10
Continue to review full report at Codecov.
|
d70f41d
to
830beaa
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
rechunker/api.py
Outdated
@@ -281,14 +311,29 @@ def _setup_rechunk( | |||
raise ValueError("Source must be a Zarr Array or Group, or a Dask Array.") | |||
|
|||
|
|||
def _validation_options(options): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: _validate_options
might read better
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @tomwhite. Let me know if there's anything else I can do to get this merged @TomAugspurger / @rabernat. |
Thanks. |
Thanks for all your contributions @eric-czech. It's great to have you involved in rechunker! |
Sure thing @rabernat! |
Fixes #46
This adds options to
rechunk
that are passed to zarr create methods. The only use cases I have in mind for this are allowing for overwrites and compression, but it may be useful in other ways. Otherwise it may eventually be worth lifting those parameters into therechunk
signature given that allowing any option to be passed is a bit of a footgun.Note: I had to modify one of the existing test fixtures since it was doing almost what I wanted for a compression test except that the chunks were too small for compression to take effect. I didn't know that occurred with zarr but I can't find any other explanation for why specifying a compressor makes no difference when an array is too small. I have no idea where the cutoff is but perhaps it is related to compression block sizes? Let me know if anybody understands that better.